Addressing scheme supporting variable local addressing and variable global addressing

ABSTRACT

A node comprises at least one agent and an input/output (I/O) circuit coupled to an interconnect within the node. The I/O circuit is configured to communicate on a global interconnect to which one or more other nodes are coupled during use. Addresses transmitted on the interconnect are in a first local address space of the node, and addresses transmitted on the global interconnect are in a global address space. The first local address space includes at least a first region used to address at least a first resource of the node. The node is programmable, during use, to relocate the first region within the first local address space, whereby a same numerical value in the first local address space and a second local address space corresponding to one of the other nodes coupled to the global interconnect refers to the first resource in the node during use.

PRIORITY INFORMATION

This application is a continuation of and claims priority to U.S. PatentApplication having an application Ser. No. 10/439,297; filed May 15,2003, which application claims benefit of priority to U.S. ProvisionalApplication Ser. No. 60/380,740, filed May 15, 2002, and in which bothare incorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention is related to address spaces in multi-node systems and,more particularly, to mapping local and global addresses.

2. Description of the Related Art

In many systems, multiple nodes are coupled together to form the system.Each node may comprise an integrated circuit, or multiple integratedcircuits and/or other devices (e.g. input/output (I/O) devices and/orinterfaces). Each node has a local address space used to address memoryin the node or coupled to the node (“local memory”), as well as variousI/O devices or interfaces in the node. Typically, the address spaces arerelatively fixed (e.g. various regions within the address space arededicated to local memory or I/O devices). There may be some amount ofprogrammability to the address space (e.g. regions of the address spacemapped to memory may be sized to permit varying amounts of localmemory).

The address space within each node of a multi-node system typicallymatches. That is, given the same amount of memory and the same I/Odevices, the same numerical addresses in each local address space referto the local memory or I/O devices in that node. Accordingly, sharinglocal memory or I/O devices with other nodes (permitting the other nodesto access and/or update the shared local memory or devices) iscomplicated. One typical mechanism is shown in FIG. 1.

In FIG. 1, a local address space 10 corresponding to a first node (node0), a local address space 12 corresponding to a second node (node 1),and an I/O address space 14 corresponding to an I/O interface used tocommunicate between node 0 and node 1 are shown. Address 0 is at thebottom of each address space in FIG. 1. Each local address space has avariety of regions (e.g. a local I/O region for the local I/O devicesand interfaces in each node, a memory region for the local memory, andan external region which is mapped onto the I/O address space 14). Whilecontiguous regions are shown in FIG. 1 for simplicity, multiple localI/O regions and/or memory regions may be defined in each local addressspace 10 and 12.

A shared memory location 16 in the node 1 local address space is alsoillustrated, via the cross-hatched box in the node 1 local address space12. A shared I/O location (e.g. corresponding to a local I/O device orinterface that is to be shared between the nodes) may be similar. Theshared memory location 16 is addressed using an address A in the node 1local address space 12. The address A cannot be used by node 0 to accessthe shared memory location 16, as the address A is in the memory regionof the local address space 10 and refers to a local memory location 18in the node 0. For node 0 to access the shared memory location 16, anaddress in the external region must be used (to cause a transaction onthe I/O interface to communicate to node 1). Thus, for example, anaddress B in the external region at the local address space 10 may beassigned to the shared memory location 16. The address B is furthermapped to an address C in the I/O address space 14, which is mapped tothe address A in the local address space 12 by the node 1 in response toreceiving the I/O transaction on the I/O interface.

In the illustrated mechanism, three different addresses (A, B, and C)are used to access the same memory location 16. If additional nodes (notshown) access the same memory location, even more addresses may be used.Such a scheme may create complexities for software executing on thesystem. For example, if a software process that accesses the memorylocation 16, and the process migrates from one node to another, theaddress used to access the memory location 16 must be recalculated. Toperform the recalculation properly, the process must be “aware” of whichnode it is running on, which may complicate the process. Some currentlyexisting software assumes that a given local address in the externalregion of the local address space is numerically equal to thecorresponding I/O address in the I/O address space (although it clearlycannot be equal to the address in the other local address space, if ashared memory location or I/O device is being accessed in another node).Such assumptions further complicate address space management. In nodesin which virtual address spaces are implemented (e.g. nodes havingprocessors), some software may even attempt to make the virtual address,the corresponding physical address in the local address space, and thecorresponding I/O address numerically equal.

The illustrated mechanism also presents difficulties if cache coherencyis to be maintained for the shared memory location. Typically, coherencyschemes rely on comparing the addresses of transactions to the cachedaddresses in a given cache. However, if each node is using differentaddresses to access the same location, comparing the addresses isinsufficient to detect an access to the same memory location as a cachedmemory location. Some multi-node cache coherent nonuniform memory access(CC-NUMA) systems use the most significant address bits as a nodeidentifier identifying the node to which the address is mapped. Suchsystems typically design the interconnect between nodes to support aglobal address space that is shared by the nodes (e.g. the “local”address spaces are merely part of the global address space that isassigned to the node).

SUMMARY OF THE INVENTION

In one embodiment, a node comprises at least one agent and aninput/output (I/O) circuit coupled to an interconnect within the node tocommunicate between the agent and the I/O circuit. The I/O circuit isconfigured to communicate on a global interconnect to which one or moreother nodes are coupled during use. Addresses transmitted on theinterconnect are in a first local address space of the node, andaddresses transmitted on the global interconnect are in a global addressspace. The first local address space includes at least a first regionused to address at least a first resource of the node. The node isprogrammable, during use, to relocate the first region within the firstlocal address space, whereby a same numerical value in the first localaddress space and a second local address space corresponding to one ofthe other nodes coupled to the global interconnect refers to the firstresource in the node during use.

A system is contemplated, in one embodiment, that comprises a globalinterconnect, a first node coupled to the global interconnect, and asecond node coupled to the global interconnect. Addresses transmitted onthe global interconnect are included in a global address space. Thefirst node has a first local address space and comprise at least oneresource addressed using a first region within the first local addressspace. Similarly, the second node has a second local address space. Thefirst node is programmable, during use, to relocate the first regionwithin the first local address space. Additionally, the second node isprogrammable, during use, to relocate regions within the second localaddress space to map a second region within the second local addressspace for transmission on the global interconnect. The second regioncomprises addresses having the same numerical value as addresses in thefirst region, whereby the resource in the first node is accessed using asame numerical value of the address in the first local address space andthe second local address space during use.

In another embodiment, a method is contemplated. A first node isconfigured to map a first region of a global address space to a secondregion of a first local address space within the first node. The secondregion is defined to address at least one resource in the first node.The first node is programmed to relocate the second region within thefirst local address space, whereby a same numerical value is used in thefirst local address space and in the global address space to address alocation in the first region.

BRIEF DESCRIPTION OF THE DRAWINGS

The following detailed description makes reference to the accompanyingdrawings, which are now briefly described.

FIG. 1 is a block diagram of a prior art mapping addresses in local andglobal address spaces.

FIG. 2 is a block diagram of one embodiment of a plurality of nodes in asystem.

FIG. 3 is a block diagram illustrating one embodiment of address spacesand mapping addresses.

FIG. 4 is a block diagram illustrating a second embodiment of addressspaces and mapping addresses.

FIG. 5 is a block diagram of one embodiment of a node shown in FIG. 2.

FIG. 6 is a block diagram of one embodiment of I/O circuitry shown inFIG. 5.

FIG. 7 is a flowchart illustrating one embodiment of configuring nodesin the system shown in FIG. 2.

FIG. 8 is a block diagram illustrating one embodiment of a computeraccessible medium.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that the drawings and detaileddescription thereto are not intended to limit the invention to theparticular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention as defined by the appendedclaims.

DETAILED DESCRIPTION OF EMBODIMENTS

Turning now to FIG. 2, a block diagram of one embodiment of a system 20including a plurality of nodes (node 0 through node 3) coupled to aglobal interconnect 22 is shown. In the illustrated embodiment, eachnode is coupled to a respective local memory 24A-24D. Each node includesa storage (26A-26D, respectively, in FIG. 2) that stores programmableregion maps. The number of nodes in a given embodiment may vary, and maybe more or fewer than the number of nodes shown in FIG. 2. Additionally,not all nodes may have local memory, in some embodiments.

Each node may have its own local address space, used to address (or“refer to”) resources within the node. Additionally, a global addressspace is employed for transactions on the global interconnect 22. Thetransactions on the global interconnect 22 may be used, for example, tocommunicate between the nodes 0 through 3.

Each node may have one or more regions in its local address space. Eachregion is assigned one or more resources of the node. That is, aresource is addressed (or referred to) using an address or addresseswithin the region to which that resource is assigned. The remainder ofthe local address space (i.e. those addresses that are not within theregions) may be defined as an external region that causes the node togenerate a transaction on the global interconnect 22. Each node isprogrammable to relocate the regions within the local address space(e.g. by programming the storages 26A-26D). Thus, the local address (oraddresses) used to address a given resource is programmable, and may bemoved about in the local address space. By relocating regions havingresources that are to be shared with other nodes, the numerical value ofthe local address used to address the resources may be made equal to thenumerical value of the local address (within the external region of thelocal address space) of the other nodes that are to share the resource.Thus, the same numerical value may be used in any local address space toaddress a resource within a particular node. Processes that may migratefrom node to node may use the same address to access a given resource ina given node. That is, if the process is executing on the given node,the access to the given resource may occur locally in that given nodebecause the address is within the region corresponding to that resource.If the process is executing on a different node, the address is withinthe external region and thus the node may generate a global transactionon the global interconnect 22 to the given node, and the given node mayaccess the resource in response to the global transaction.

The nodes may also be programmed to make one or more resources private(i.e. not shared). If the same region in each node is programmed to thesame address range within each local address space, then resourceswithin those regions will be private to the local node.

Additionally, in some embodiments, the numerical value of the localaddress may be made equal to the global address mapped to the resource.Generally, there may be a mechanism for exposing regions that areaccessible via transactions on the global interconnect 22, and theseregions may be programmed during initialization of the system 20 to aregion of the global address space. For example, in some embodiments,the global interconnect 22 may be compatible with peripheral componentinterconnect (PCI) specification or the HyperTransport™ (HT)specification. These specifications use special configuration commandsto access configuration header data structures for each device on theinterface. The configuration header stores information identifying thedevice, its functions, etc. The size of an address range used to addressthe device may also be identified. The beginning of the address rangemay be programmable in a base address register (BAR), thus permitting aregion of the global address space to be assigned to the device. A givendevice may have multiple address ranges assigned in this manner. Thus,for example, a node may have an address range corresponding to eachrelocatable region, and the BAR for the region may be programmed duringinitialization. Nodes may have additional address ranges/BARs for otherpurposes as well. In other embodiments, the numerical value of theglobal address may differ from the numerical value of the correspondinglocal addresses.

In some embodiments, internode coherency may be maintained for memoryresources that are shared between nodes. For example, a CC-NUMA schememay be used. Since the same numerical value of addresses in each localaddress space are used to access the shared memory resource, theaddresses may be compared to maintain coherency between accesses byvarious nodes. Some memory resources (e.g. memory storinginterrupt/exception vectors, or the memory storing the boot read-onlymemory (ROM) or programmable ROM (PROM)) may be maintained private (andnon-coherent at the internode level), and other memory regions may beshared.

As used herein, a “node” may comprise any set of one or more integratedcircuits, I/O devices or interfaces, and other circuitry which arearranged together and interact using a local address space. An “addressspace” may comprise a set of addresses which are used to communicatebetween various components. A “local address space” may be used tocommunicate within a node, and a “global address space” may be used tocommunicate between nodes. Addresses in different address spaces may notnecessarily have any relationship to each other. However, as mentionedabove, the storages 26A-26D may be programmed to make certain addresseswithin the address spaces be numerically equal. A “region” of an addressspace comprises a contiguous range of addresses within the addressspace. A relocatable region may be programmable as to where region islocated in the address space. A node may have one or more resources thatare addressable via addresses in the address space. More particularly,each resource may be assigned to one of the regions defined in theaddress space. As used herein, the term “resource” may generally referto any device or circuitry that may be used to provide a function. Theterm resource may include portions of the local memory (e.g. the memory24A for node 0) and/or I/O devices or interfaces in the node. The memorymay provide a storage function, and the I/O devices or interfaces mayprovide any desired peripheral function.

The storages 26A-26D may be programmed to map the region in the localaddress spaces, as mentioned above, and may further include storage forthe BAR programming mentioned above for the global interconnect 22.Generally, the storages 26A-26D may be implemented as any sort ofstorage device. For example, the storages 26A-26D may be implemented asregisters or other clocked storage devices, memory such as random accessmemory (RAM), content addressable memory (CAM), or combinations of theabove storage devices. Each storage 26A-26D may be distribute among thenode components within the corresponding node, including duplicatingstorage among the node components as desired.

The global interconnect 22 may be any form of communication mediumbetween the nodes. For example, in various embodiments, the interconnect22 may include shared buses, crossbar connections, point-to-pointconnections in a ring, star, daisy-chain, or any other topology, meshes,cubes, etc. The global interconnect 22 may also include storage, in someembodiments. In one particular implementation, the global interconnect22 may comprise point-to-point connections for packet-basedcommunication as defined in the HT specification. In anotherimplementation, the global interconnect 22 may comprise a shared buscompatible with the PCI specification. Other implementations may useother industry-standard I/O interconnects or a custom-designedinterconnect, as desired. A transaction transmitted on the interconnectmay include a communication sourced by one of the nodes on theinterconnect and targeting at least one other node on the interconnect.Some transactions may target more than one node (e.g. a broadcast ormulticast transaction). The transaction includes an address in theglobal address space, and may include a transfer of data (e.g. a read orwrite transaction). If a coherency scheme (e.g. CC-NUMA) is implementedfor internode coherency, transactions may also include transactions tomaintain coherency (e.g. probe commands, acknowledgements of the probecommands, data transfers, etc.)

Each of the memories 24A-24D may comprise any type of memory device. Forexample, the memories 24A-24D, in various embodiments, may includesynchronous dynamic RAM (SDRAM), and more particularly double data rate(DDR) SDRAM, DRAM, DDR synchronous graphics RAM (SGRAM), DDR fast cycleRAM (FCRAM), DDR-II SDRAM, Rambus DRAM (RDRAM), SRAM, or any othersuitable memory device or combinations of the above mentioned memorydevices.

FIG. 3 is a block diagram illustrating one embodiment of the localaddress spaces for nodes 0 and 1 (reference numerals 40 and 42,respectively) and a global address space 44 corresponding to the globalinterconnect 22. Similar to FIG. 1, address 0 in each space is shown atthe bottom of the space.

In the illustrated embodiment, there are four relocatable regions in thelocal address spaces, labeled “Mem 0”, “I/O 0”, “Mem 1”, and “I/O 1”. Inthis embodiment, each region has an associated base address pointer,labeled “M0BA” (Mem 0 base address), “I0BA” (I/O 0 base address), “M1BA”(Mem 1 base address), and “I1BA” (I/O 1 base address), respectively. Mem0 and Mem 1 are memory regions (each mapped to a portion of the localmemory in the corresponding node). I/O 0 and I/O 1 are I/O regions, eachmapped to one or more I/O devices or interfaces in the node. Theremainder of the local address spaces 40 and 42 are “external”. The noderesponds to a transaction having an external address by generating aglobal transaction on the global interconnect 22.

In the node 0 local address space 40, the relocatable regions areprogrammed to the low end of the local address space. This may also bethe default location of the regions, for one embodiment (i.e. if noprogramming is performed, the regions are located as shown for the node0 local address space 40).

In the node 1 local address space 42, the relocatable regions areprogrammed to numerical values that correspond to the external addressregion in the node 0 local address space 40. Similarly, the low end ofthe node 1 local address space 42 is part of the external region andthus the relocatable regions of the node 0 local address space 44 areprogrammed to numerical values that correspond to the external addressregion in he node 1 local address space 42. In this example, each of therelocatable regions of the local address space 40 are accessible usingthe same numerical addresses in the node 1 local address space 42, andvice versa.

For example, an access to the location 46 in the Mem 0 region may use anaddress A. If the access to address A is performed by an agent in thenode 0, the access may be a local access to the local memory 24A that ismapped to the Mem 0 region. No activity on the global interconnect 22may occur (unless the location 46 is coherent and coherency activitysuch as probes are needed). If the access to address A is performed byan agent in the node 1, the node 1 recognizes address A as beingexternal and transmits a global transaction on the global interconnect22. The transaction is mapped through the BAR corresponding to the Mem 0region in the node 0 (labeled N0M0 BAR in FIG. 3) to the location 46.Thus, the same numerical address may be used in both nodes 0 and 1 toaccess the location 46.

Similarly, an access to the location 48 in the Mem 1 region of the node1 local address space 42 may use an address B. If the access to addressB is performed by an agent in the node 1, the access may be a localaccess to the local memory 24B that is mapped to the Mem 1 region. Noactivity on the global interconnect 22 may occur (unless the location 48is coherent and coherency activity is needed). If the access to addressB is performed by an agent in the node 0, the node 0 recognizes addressB as being external and transmits a global transaction on the globalinterconnect 22. The transaction is mapped through the BAR correspondingto the Mem 1 region in the node 1 (labeled N1M1 BAR in FIG. 3) to thelocation 48. Thus, the same numerical address may be used in both nodes0 and 1 to access the location 48.

As alluded to above, each relocatable region may have an associated BARto map the region in the global address space 44. Thus, in this example,there are 4 BARs for the 4 relocatable regions in node 0: “N0M0 BAR”corresponding to the Mem 0 region in node 0; “N0I0 BAR” corresponding tothe I/O 0 region in node 0; “N0M1 BAR” corresponding to the Mem 1 regionin node 0; and “N0I1 BAR” corresponding to the I/O 1 region in node 0.Similarly, there are 4 BARs for the 4 relocatable regions in node 1:“N1M0 BAR” corresponding to the Mem 0 region in node 1; “N1I0 BAR”corresponding to the I/O 0 region in node 1; “N1M1 BAR” corresponding tothe Mem 1 region in node 1; and “N1I1 BAR” corresponding to the I/O 1region in node 1. Additional BARs may be defined for relocatable regionsin other nodes.

Additionally, in this example, the relocatable regions have been alignedto the numerical address in their corresponding BARs. In so doing, thesame numerical address is also used on the global interconnect 22 toaccess a shared location. Additionally, the example of FIG. 3 does notshow any non-relocatable regions. However, embodiments are contemplatedin which one or more regions are provided that are not relocatable.These regions may have fixed locations in the local address space, andthe relocatable regions may not be programmed to overlap with thenon-relocatable regions. The non-relocatable regions also may not, ifdesired, be represented in the programmable region maps in the nodes.Since they are fixed, the non-relocatable regions may be decoded bydedicated decode circuitry. Alternatively, the non-relocatable regionsmay be represented in the programmable region maps as well, to use thesame mechanism for all addresses.

Each of the relocatable regions are shared in the example of FIG. 3.However, it may be desirable to share some regions and not others. FIG.4 is an example of sharing the Mem 1 and I/O 1 regions of each of thelocal address spaces 40 and 42, but keeping the Mem 0 and I/O 0 regionsof each of the local address spaces 40 and 42 private. Accordingly, theMem 0 and I/O 0 regions in nodes 0 and 1 are programmed to the samenumerical addresses within their respective local address spaces 40 and42. The Mem 1 and I/O 1 regions are programmed to addresses in theirlocal address spaces 40 and 42 which are in the external region of theother local address space 42 and 40.

Therefore, an access to an address within the Mem 0 or I/O 0 regions byan agent in node 0 is completed locally in node 0. Similarly, an accessto an address within the Mem 0 and I/O 0 regions by an agent in node 1is completed locally in node 1. An agent in node 1 may not access theMem 0 or I/O 0 regions of node 0, and an agent in node 0 may not accessthe Mem 0 or I/O 0 regions of node 1. On the other hand, a location 50in the Mem 1 region of the node 0 local address space 40 may be accessedusing an address C in either of the local address spaces 40 and 42, andthe location 52 in the Mem 1 region of the node 1 local address space 42may be accessed using an address D in either of the local address spaces40 and 42, similar to the discussion above with regard to locations 46and 48 in FIG. 3.

In addition to programming the Mem 0 and I/O 0 regions to the samenumerical value within their respective local address spaces, the nodes0 and 1 may not expose the corresponding BARs (N0M0 BAR, N0I0 BAR, N1M0BAR, and N1I0 BAR) to the initialization code that maps BARs into theglobal address space. Thus, there is no mapping of the Mem 0 and I/O 0regions in the global address space 44 in FIG. 4.

While FIGS. 3 and 4 illustrate two local address spaces, there is alocal address space for each node in the system 20. The relocatableregions may be programmed to addresses that numerically match theexternal region of each other local address space, or the externalregion of at least those local address spaces with which sharing isdesired.

Turning next to FIG. 5, a block diagram of one embodiment the node 0shown in FIG. 2 is shown. Other nodes 1-3 may be similar. In theembodiment of FIG. 5, the node 0 includes one or more agents (e.g.agents 30A-30B in FIG. 5), a memory controller 32 for coupling to thememory 24A, I/O circuitry 34, and an interconnect 36. The agents30A-30B, the memory controller 32 and the I/O circuitry 34 are coupledto the interconnect 36. The I/O circuitry is further coupled to theglobal interconnect 22, and may optionally be coupled to one or moreother I/O interconnects. As illustrated in FIG. 5, the agents 30A-30B,the memory controller 32, and the I/O circuitry 34 include one or morestorages 26AA-26AD that store local address (LA) region maps.Additionally, the I/O circuitry 34 includes one or more storages 26AEthat store LA to global and global to LA maps. The storages 26AA-26AEmay be portions of the storage 26A shown in FIG. 2.

The agents 30A-30B may comprise devices which communicate on theinterconnect 36. Generally, as used herein, an agent may be any devicecapable of communicating on the interconnect 36. For example, agents maycomprise processors, caches, etc. The memory controller 32 and the I/Ocircuitry 34 may also be agents. Some agents may be capable ofinitiating transactions (source agents), and other agents may be capableof receiving transactions (target agents). Some agents may be capable ofbeing both source and target agents. For example, processors may besource agents. The memory controller may be a target agent fortransactions addressing the memory 24A. The I/O circuitry 34 may be asource agent for transactions generated in response to globaltransactions from the global interconnect 22, and may be a target agentfor transactions that address I/O devices or are in the external regionmapped to the global interconnect 22. Other agents may participate in agiven transaction (e.g. a coherent transaction may have a variety ofparticipating agents, including any agents that may cache data).

Each of the agents 30A-30B, the memory controller 32, and the I/Ocircuitry 34 is programmed with the LA region maps. The LA region mapsare programmed to relocate the various regions defined in the localaddress space, and the remaining portion of the local address space maybe mapped as external. Thus, the agents 30A-30B, the memory controller32, and the I/O circuitry 34 may access the LA region maps to determinethe region in which a given address is included (either an address thatthe agent is to sourced in a transaction or an address observed by theagent in a transaction on the interconnect 36). For a source address,the source agent may determine the target of the transaction (which mayaffect the type of transaction transmitted, its attributes such ascacheable vs. noncacheable, etc.). For a received address, determiningthe region may permit the agent to determine if it is to respond to thetransaction (i.e. to determine if it is the target of the transaction).For example, the memory controller 32 and the I/O circuitry 34 maydetermine whether or not to respond to a transaction using the LA regionmaps. The memory controller 32 responds to memory regions and the I/Ocircuitry 34 may respond to the other regions (e.g. I/O regions or theexternal region).

In some embodiments, the LA region maps in a given agent may include aninbound region map and an outbound region map. The outbound region mapmay be used by a source agent for transactions that the agent ispreparing to source on the interconnect 36. The outbound region map maybe used in a variety of ways. For example, in one embodiment, flowcontrol is implemented on the interconnect 36 using a source-blockingscheme. That is, each agent may signal whether or not it is currentlyable to participate in a transaction. A source agent may determinewhether or not to initiate a transaction dependent on whether the targetagent and any other participating agents are signaling ability toparticipate in the transaction. The source agent may determine whichagent is the target (e.g. the memory controller 32 or the I/O circuitry36) by looking up the transaction address using the outbound region map,and also may determine other participating agents (e.g. dependent onwhether the transaction is coherent or not).

The inbound region map may be used by potential target agents todetermine if the transaction on the interconnect 36 is targeted at thatagent. In some cases, an agent may be a proxy for indicating ability toparticipate in transactions for another agent. In such cases, the proxyagent may use the inbound region map to compute whether or not the otheragent is able to participate in transactions. Generally, both theoutbound region map and the inbound region map may receive a localaddress as an input, and may output an indication of the region that theaddress is included in. In some cases, if the agent maps the localaddress to a different address internally, the inbound region map mayoutput information used to map the local address to the internaladdress.

Since the region maps are distributed to the various agents in the node0, a mechanism to synchronize updates to the maps may be implemented.For example, a broadcast transaction on the interconnect 36 may be usedto provide the contents of the region maps to each of the storages26AA-26AD. The broadcast may be implemented, for example, as a writetransaction to a predetermined “well-known” address, and the data forthe write transaction may be the updated content for the region maps.Each agent 30A-30B, 32, and 34 may receive the content and update the LAregion maps accordingly.

The region maps may represent the mapping of the regions in any desiredfashion. For example, in one embodiment, each region may have a baseaddress pointer identifying the base address (a local address in thelocal address space) of the region. The region maps may store thepointers, or may represent the region in any other desired fashion.

The I/O circuitry 34 further includes LA to global and global to LAmaps. The LA to global map may be used to map local addresses that arein an I/O region or the external region to global addresses for a globaltransaction to be initiated on the global interconnect 22. Similarly,the global to LA map may be used to map global addresses (from globaltransactions received on the global interconnect 22) to local addresses.Generally, the LA to global and global to LA maps may reflect the BARaddresses programmed into the node 0 during initialization of the system20. The maps may include the BARs, or may represent the informationprogrammed into the BARs in some other fashion. In some cases (e.g. therelocatable regions), the local and global addresses may be numericallyequal. In other cases, the local and global addresses may differ.Additionally, there may be other types of regions (e.g. match bit laneand match byte lane regions for PCI or HT transactions, special accessregions such as configuration regions, etc.) in some embodiments.

The interconnect 36 may be any type of communication medium. Forexample, the interconnect 36 may be any of the types of interconnectgiven above for the global interconnect 22. In one particularembodiment, the interconnect 22 may comprise a split transaction bus.Source agents may transmit transactions on the interconnect 36,including addresses in the local address space of the node.

The memory controller 32 may be configured to access any type of memory.Fro example, the memory controller 32 may be configured to access any ofthe various types of memory described above with regard to FIG. 2 forthe memories 24A-24D.

The I/O circuitry 34 may include circuitry for communicating on theglobal interconnect 22 and the interconnect 36, and for initiatingtransactions on one interconnect in response to being a target of atransaction on the other interconnect. The I/O circuitry 34 may alsoinclude various local I/O devices or interfaces for the node 0 (that is,the local I/O devices and interfaces that are assigned to one or morerelocatable I/O regions in the local address space of the node 0). Anexample of one embodiment of the I/O circuitry 34 is shown in FIG. 6 anddescribed in more detail below.

It is noted that, in some embodiments, a node may define one or moreregions in the local address space that are not relocatable, if desired.In one embodiment, the node 0 may be integrated onto a single integratedcircuit as a system on a chip (SOC) configuration.

Turning now to FIG. 6, a block diagram illustrating one embodiment ofthe I/O circuitry 34 is shown. In the illustrated embodiment, the I/Ocircuitry 34 includes a first bridge 60, a second bridge 62, a switch64, a set of HT interface circuits 66A-66C, a PCI interface circuit 68,and other local I/O circuitry 70. The bridges 60 and 62 are coupled tothe interconnect 36. The bridge 60 is further coupled to the switch 64,which is still further coupled to the HT interface circuits 66A-66C.Each of the HT interface circuits 66A-66C, each of which are coupled toHT interfaces which form a portion of the global interconnect 22. Thebridge 62 is further coupled to the PCI interface circuit 68 (which isfurther coupled to a PCI bus) and other local I/O circuitry 70 (whichmay optionally be coupled to other I/O interfaces). Both of the bridges60 and 62 may be coupled to the LA regions maps storage 26AD as well.

The first bridge 60 may be responsible for bridging transactions betweenthe interconnect 36 and the global interconnect 22 (which in thisembodiment may comprise HT interfaces). Thus, the bridge 60 may includeLA to global and global to LA maps (stored in a storage 26AE) to maplocal addresses to global addresses and vice-versa. Additionally, thefirst bridge 60 may be coupled to the storage 26AD to access the LAregion maps, to detect those addresses that are in the external regionof the local address space. The bridge 60 may generate HT transactionsin response to local transactions on the interconnect 36 fortransmission on the global interconnect 22. The HT transactions aretransmitted, through the switch 64, to the HT interface circuit 66A-66Ccoupled to the HT interface on which the HT transaction is to betransmitted. The HT interface is a set of point-to-point links to otherdevices (e.g. other nodes) and thus which interface the HT transactionis to be transmitted on is dependent on the receiving device. The LA toglobal maps may store an indication of which interface to use, oranother mechanism may be used to identify the interface on which the HTtransaction is to be transmitted. Additionally, HT transactions receivedby the HT interfaces may be passed, through the switch, to the bridge 60if the target is the node 0. There may be enough information in thereceived HT transaction to determine if the target is node 0, or the HTinterface circuits 66A-66C may include storage for a routing map used todetermine if the target is node 0, or which interface to route thereceived HT transaction on if the target is not node 0.

It is noted that, while 3 HT interface circuits 66A-66C are shown inFIG. 6, other embodiments may include any number of HT interfacecircuits 66A-66C. In some embodiments, the HT interface circuits 66A-66Cmay be programmable as either HT interface circuits or as System PacketInterface (SPI) interface circuits (e.g. SPI-4). It is noted that, whilethe switch 64 is used to transfer data between the bridge 60 and the HTinterface circuits 66A-66C, other embodiments may not include the switchand may directly couple the HT interface circuits 66A-66C and the bridge60.

The second bridge 62 may be responsible for bridging transactionsbetween the interconnect 36 and the various local I/O devices orinterfaces. An I/O device or interface is local if it is not used tocommunicate with other nodes in the system 20. Thus, for example, thePCI interface circuit 68 may be used to couple the node 0 to variousperipheral devices, and may be a local I/O interface. The other localI/O circuits may include various devices and interfaces (e.g. a genericbus for access to a boot PROM, serial and/or parallel ports, personalcomputer memory card interface (PCMCIA) ports, one or more Ethernetmedia access controllers (MACs), etc.). In some cases, the localaddresses are mapped to local I/O addresses and vice versa, and thus astorage 26AF may be included to map the addresses. For example,transactions received on the PCI bus may be mapped through one or morePCI BARs to local addresses. Any desired mapping mechanisms may be used.Additionally, since one or more of the local I/O devices/interfaces maybe included in the relocatable I/O regions in the local address space,the second bridge 62 may be coupled to the storage 26AD to access thelocal address region maps.

Turning next to FIG. 7, a flowchart is shown illustrating configurationof the nodes in a system 20. The flowchart of FIG. 7 may be implementedas software instructions which, when executed (e.g. on a processorwithin one of the nodes, for example), implement the functions shown inFIG. 7.

The configuration software may configure the global address space(assigning each nodes BARs, including BARs corresponding to therelocatable regions of the local address space in each node) (block 80).Block 80 may comprise instructions which, when executed, performconfiguration accesses on the global interconnect 22 to detect all ofthe exposed BARs, and to assign them in the global address space.

Once the BARs have been assigned, the configuration software may programthe LA region maps in each node to relocate the regions to addressesthat numerically match the assigned BARs in the global address space(block 82). Thus, the global addresses and the local addresses in anylocal address space that are used to access a given region may benumerically equal.

Turning next to FIG. 8, a block diagram of a computer accessible medium300 including one or more data structures representative of thecircuitry included in the node 10 and/or the configuration software 302that implements the flowchart of FIG. 7 is shown. Generally speaking, acomputer accessible medium may include any media accessible by acomputer during use to transfer data and/or instructions to/from thecomputer. The computer accessible medium may include storage media suchas magnetic or optical media, e.g., disk (fixed or removable), CD-ROM,or DVD-ROM, various recordable or rewritable CD or DVD formats, volatileor non-volatile memory media such as RAM (e.g. SDRAM, RDRAM, SRAM,etc.), ROM, etc., as well as media accessible via transmission media orsignals such as electrical, electromagnetic, or digital signals,conveyed via a communication medium such as a network and/or a wirelesslink.

Generally, the data structure(s) of the circuitry on the computeraccessible medium 300 may be read by a program and used, directly orindirectly, to fabricate the hardware comprising the circuitry. Forexample, the data structure(s) may include one or more behavioral-leveldescriptions or register-transfer level (RTL) descriptions of thehardware functionality in a high level design language (HDL) such asVerilog or VHDL. The description(s) may be read by a synthesis toolwhich may synthesize the description to produce one or more netlist(s)comprising lists of gates from a synthesis library. The netlist(s)comprise a set of gates which also represent the functionality of thehardware comprising the circuitry. The netlist(s) may then be placed androuted to produce one or more data set(s) describing geometric shapes tobe applied to masks. The masks may then be used in various semiconductorfabrication steps to produce a semiconductor circuit or circuitscorresponding to the circuitry. Alternatively, the data structure(s) oncomputer accessible medium 300 may be the netlist(s) (with or withoutthe synthesis library) or the data set(s), as desired. In yet anotheralternative, the data structures may comprise the output of a schematicprogram, or netlist(s) or data set(s) derived therefrom.

While computer accessible medium 300 includes a representation of thenode 0, other embodiments may include a representation of any portion ofthe node 0 (e.g. agents 30A-30B, memory controller 32, I/O circuitry 34,etc.). Additionally, the computer accessible medium 300 includes theinstructions forming the configuration software 302 which, whenexecuted, implement the flowchart of FIG. 7.

Attached hereto is an Appendix A comprising a description of variousdetails that may be implemented in one embodiment of a system on a chipthat may implement various features described above. Appendix A forms apart of this detailed description and is incorporated herein in itsentirety.

Numerous variations and modifications will become apparent to thoseskilled in the art once the above disclosure is fully appreciated. It isintended that the following claims be interpreted to embrace all suchvariations and modifications.

1-19. (canceled)
 20. A method comprising: configuring a first node tomap a first region of a global address space to a second region of afirst local address space within the first node, the second regiondefined to address at least one resource in the first node; andprogramming the first node to relocate the second region within thefirst local address space, whereby a same numerical value is used in thefirst local address space and in the global address space to address alocation in the first region.
 21. The method as recited in claim 20wherein configuring the first node comprises programming a globaladdress to local address map storage in the first node to map the secondregion to the first region.
 22. The method as recited in claim 20further comprising programming a second node to map one or more regionsin a second local address space of the second node such that a thirdregion having a same numerical value as the second region and the firstregion is mapped to the global address space.